Voronoi Methods for Assessing Statistical Dependence

In this work, we leverage the intuitive geometry of the Voronoi diagram to develop a new class of methods for quantifying and testing for statistical dependence between real-valued random variables. The contributions of this work are threefold: First, we introduce the utility of probability integral transforms of the datapoints to produce Voronoi diagrams which exhibit desirable properties provided by the theory of copula distributions. Second, we propose a novel nonparametric statistical test (VORTI) to determine whether two variables are independent based on the areas of the cells in the Voronoi diagram. Based on similar ideas, we finally introduce a new method (SVORMI) for estimating mutual information which relies on a novel regularization technique we found no mention of in the literature on density estimation via Voronoi tessellation. Through extensive simulation experiments, we demonstrate that our proposed estimator is competitive with the state-of-the-art k-nearest neighbor mutual information estimator of Kraskov et al. (2004). Introduction Detecting various forms of deviations from statistical independence between random variables is a cornerstone of statistics. A wide range of approaches to this problem have been explored, ranging from correlation coefficients based on data values/ranks/distances [Pearson (1896); Hazewinkel (2001); Kendall (1938); Szekely and Rizzo (2009)], Fourier or RKHS representations [Stein and Weiss (1971); Gretton et al. (2005, 2007)], and informationtheoretic tools [Cover and Thomas (2006)]. Each class of methods has its advantages and disadvantages and relies on different assumptions on relationships present in the data. An extremely popular measure of dependence in machine learning is the mutual information (see Definition 1). Intuitively, mutual information (MI) measures the degree to which knowing the value of one variable gives information about another. It has several properties that make it a desirable basis for a measure of dependence: it is non-negative and symmetric, it is invariant under order-preserving transformations of random variables X and Y , and, most importantly, it equals zero precisely when X and Y are statistically independent. These properties—which are not shared by simpler measures of correlation such as Pearson correlation—have led to its widespread use in diverse applications [Elemento et al. (2007)].